Date: Mon, 11 Jun 90 18:40:53 PDT
From: psz@sumex-aim.stanford.edu (Peter Szolovits)
Subject: Summary of advice on HST modems

Several days ago, I asked for help figuring out how to run my US Robotics
Courier HST modem more efficiently for communication between my home Mac and a
Unix box at the office (via a networked terminal concentrator).  The summary
results are short:

1.  Use zmodem rather than kermit, to avoid various delays in kermit
handshaking (even with 1000-byte packets).  info-mac/comm/zterm-085.hqx
contains a shareware communication program that supports zmodem, and I was also
pointed to the latest White Knight and MicroPhone, both commercial products.
At the Unix end, you need an implementation of zmodem, which is available in
info-mac/unix/zmodem-part*.hqx.  (I had to get rid of a couple of spurious
newlines in the Makefile to get it to make.)  The ZTerm/zmodem combination does
indeed give me about 93% of theoretical max line utilization on large text
files at 9600 baud.  I've had a bit of trouble uploading MacBinary II files,
though not consistently.  For straight text dump, ZTerm can't keep up with
scrolling at 9600 baud, so the buffer overruns and you lose captured data on a
long file without flow control.  Zmodem transfer seems to work fine without
flow control; I guess putting characters on the screen is what's much slower
than capturing to a file, even having to do CRC checks and all.

2.  Going from 9600 baud to 19,200 baud is highly "non-trivial", and although
everyone thinks it can be done, I'm not sure if any of the respondents have
actually done it routinely in ways that I could duplicate.  First of all, flow
control is essential because the computers have license to send bits faster
than the line can deliver them if compression fails to achieve 50%.  ^S/^Q
(XON/XOFF) flow control is relatively easy to set up, but then interferes with
the ability to send the full ASCII character set (worst, for me, is that it
conflicts with Emacs ^S).  The HST modems also support hardware flow control
via the RTS/CTS lines, but the normal Mac/modem cable does not connect these.
Apparently the Mac only has one set of handshake lines, so the cable has to be
custom-made to connect those to RTS/CTS at the modem; then you lose DTR and
whatever the other line is now used for (CD or RI? for auto-answer?).  Second,
many dial-ups are not set up for different computer/modem and modem/line data
rates, so they only allow 9600 baud.  This can be fixed by politely asking (or
bludgeoning) a system administrator, I'm told.  Third, you have to play with
the communication parameters of whatever the modem is hooked to at the other
end from your Mac.  This means either stty in Unix or the command language of
your concentrator box.  Great mysteries abound here, though these too are, in
principle, solvable.

Attached is the text of my correspondence with several VERY helpful people,
whom I would like to thank: Marek Lugowski, Dave Platt, Michael Hoffhines,
Charles E. Bess, Warren Burton, Christopher Owens, Chris Eliot, Thomas Wu
Dave Bursik, and Veljko Roskar, in chronological order.

--Peter Szolovits

----------------------------------------------------------------
Moderators:
	You may want to post the following under tips for others who want to
dive into the details.  --Pete Szolovits


 7-Jun-90  0:59:11-GMT,4351;000000000000
Date: Wed, 6 Jun 1990 17:59:11 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: Info-Mac@sumex-aim.stanford.edu
Subject: Efficient use of Courier HST modems 
Message-ID: <CMM.0.88.644720351.psz@sumex-aim.stanford.edu>

I have been frustrated by my inability to squeeze optimal performance out of
our US Robotics Courier HST modems with any of the communication packages I
have tried (or heard of) and wondered if anyone has found a reasonable solution
to the identical problem.  Usually, in addition to some sort of terminal
emulation, for which almost any program I've looked at does a reasonable job
(e.g., Kermit is just fine), I want to do the fastest possible up and
downloads.  For a download, this means going from a Unix box over ethernet to a
network terminal concentrator like a CISCO box with these Courier HST modems,
over a phone line to my Mac, with the same kind of modem.  Upload just reverses
the path.  These modems (about two years old now) support 9600 baud with USR's
own signaling scheme (roughly, it's a rapid-turnaround asymmetrical allocation
of bandwidth, 9600/300, so even though it looks like full duplex, there's a
fast channel and a slow channel that get swapped by the modems as they detect
traffic volume).  In addition, these modems support error correction and the
ability to do data compression, yielding, in principle, communication rates
close to 19.2 kilobits/sec.

Transferring an HQX file, for example, should at best give me close to 2000
bytes/sec, at the 19.2K transfer rate.  In fact, I count myself lucky if I get
anywhere near a fifth of that.  Why?

1.  Looking at the "receive data" and "send data" lights, I see that most of
the time is spent waiting for handshakes.  Using 1000-byte Kermit packets, for
example, I can see that the whole packet comes in one short burst (sometimes
with a noticable short gap in the middle, if the Ethernet packetization between
the Unix box and the terminal concentrator is slow).  Then there's at least an
equal-length delay while the acknowledgement goes back.  Clearly, the situation
is better than if we still had only 94-character (or XMODEM's 128-character)
packets, but as baud rates rise, more and more of the time goes into waiting
for the handshake.  I've read about sliding-windows Kermit and other such
attempts to get around this particular problem, but as far as I can tell this
hasn't made it into any of the programs I own (Kermit) or have been able to get
my hands on to test (Versaterm Pro 2.20, Red Ryder 9.4, Smartcom II,
MacTerminal 2.2).

2.  In order to use data compression or automatic error correction, you must be
able to support flow control.  The Courier modems support either XON/XOFF flow
control or hardware flow control (RTS/CTS lines in RS-232).  Hardware flow
control is clearly better because it doesn't interfere with use of the full
character set.  For example, I like to use Emacs, which binds Control-S as the
search key.  Also 8-bit downloads mean I can send binary files, if all the
signalling can be out-of-band.  Alas, no comm program I've seen supports
RTS/CTS flow control.  There was a recent info-mac posting of multistation-320,
which supports CTS/DTR handshake, but that program does not include file
transfers, and also DTR is the "wrong" signal.  Historically, it tells the
modem whether the attached device is up or down, not free or busy.  With the
Courier HST, resetting it causes the modem to drop the connection (at least
when in error-correcting mode).

3.  With the error correction and retry built into these modems, it seems like
the protocols used by the various communication programs should be redundant.
My experience with Kermit file transfers is that with ARQ on (Automatic Retry
Request -- the modem's error-correction feature), Kermit never sees any errors,
though I suppose it could time out if the phone line were so noisy that the
modem's own protocol could not get a Kermit packet through in the allowed time.

Is there actually a workable and closer to optimal solution that I am just
overlooking?  Do I simply have to settle for hour-long downloads that should be
achievable in less than 15 minutes?  Thanks for any information, and I'll be
happy to post to the net if there's good news.

-- Peter Szolovits
MIT Lab for Computer Science
(this year Stanford Medical Computer Science)
 7-Jun-90 23:48:39-GMT,3202;000000000011
Return-Path: <marek@iuvax.cs.indiana.edu>
Received: from iuvax.cs.indiana.edu by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA16685; Thu, 7 Jun 90 16:48:36 PDT
Message-Id: <9006072348.AA16685@sumex-aim.stanford.edu>
Received: by iuvax.cs.indiana.edu 
Date: Thu, 7 Jun 90 18:46:11 -0500
>From: Marek Lugowski <marek@iuvax.cs.indiana.edu>
To: psz@sumex-aim.stanford.edu
Subject: HST modems, fellow user's comments
Cc: marek@iuvax.cs.indiana.edu

Professor Szolovits,

I use an HST from home and our departmental dial-in machine, iuvax,
uses them also.  I live in a forest, served by arguably the country's
least avantgarde rural phone utility, Smithsville Telephone.  My
drive-in distance is 17 miles, which may be a good upper bound on the
telephone traffic.  The as-the-crow-flies distances is 7 miles, an
unrealistic lower-bound, given Lake Monroe in the way.

In short, with shareware ZTerm (by David Alverson, who reads info-mac
I beleive) (archived at your sumex-aim.stanford.edu,
~ftp/info-mac/comm) I get 940 cps in zmodem, downloading, on average,
for files 50K and longer.  It takes a while to get to that speed (no
CRC errors). 948 cps is the highest I've noticed.  This is with my HST
set to 9600 baud, 19.2k or 38.4k, no hardware control (I believe the
modems recognize each other and control accordingly; my incantation:
at e1 m1 v1 &b1 &h1 &n0 dt...).  The higher baud settings are of not
much consequence to me in downloading from iuvax.  On the other hand,
anything but 9600 is bad news for uploads because HST winds up and
delivers a pitch that is just too fast for the communication link and
ends up retrying at lower pace...  I get nearly 900 cps uploading, when
iuvax is not very busy.

In short, ZTerm gives me a reasonable vt100 emulation to other HSTs,
and zmodem, xmodem, y-modem and text transfers (but no Kermit),
important interrupts and controls, convenient two-click startup
document, scripting, buffered keyboard, reasonably convenient menus
and command-shortcuts, some basic color customization (no color-picker
or font control but the default is pleasing, black on yellow) as well
as macros.  The Zmodem is very convenient in automatically sending an
"rz" command to the iuvax and automatically starting its own process
on the mac given an "sz".  Minimal fuss.  All of this in pre release
1.0 (0.85)!  ZTerm is pretty orthodox in not wanting to bind the
control- key to anything else (such as command-) which really hurts me
in GNU Emacs on my tactilely beautiful / layout-miserable Datadesk
MAC-101 keyboard (one control key in the lower-right corner of the
strike zone; any ideas on surgically rebinding the free-motion
caps-lock?).

Hope this helps.  Please feel free to quote and hope you uncover better
news.   These HST modems are wonderful, survive accidental phone cradle
upheavals without a single protest, look like ravens, etc...

I would like to do twice as well as 948 cps but I take it this is
already twice as nice as what you have been able to get, and I do live
in the woods.

				-- Marek

Marek Lugowski
Artificial Life Research Group
Computer Science Department
Lindley Hall 101
Indiana University
Bloomington, Indiana 47406

marek@iuvax.cs.indiana.edu

 7-Jun-90 23:54:07-GMT,6007;000000000011
Return-Path: <coherent!dplatt@ames.arc.nasa.gov>
Received: from ames.arc.nasa.gov by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA16794; Thu, 7 Jun 90 16:54:03 PDT
Received: by ames.arc.nasa.gov (5.61/1.2); Thu, 7 Jun 90 16:51:36 -0700
Received: from improper.coherent.com by coherent.com (4.1/SMI-3.2)
	id AA18939; Thu, 7 Jun 90 16:42:16 PDT
Received: by improper.coherent.com (4.0/SMI-3.2)
	id AA13399; Thu, 7 Jun 90 16:42:16 PDT
Message-Id: <9006072342.AA13399@improper.coherent.com>
>From: dplatt@coherent.com (Dave Platt)
Date: Thu, 7 Jun 90 16:42:14 PDT
X-Mailer: Mail User's Shell (7.0.0 12/10/89)
To: psz@sumex-aim.stanford.edu
Subject: Re: High speed modem use with Mac telecom packages
Cc: Info-Mac@sumex-aim.stanford.edu

I've been able to use U.S. Robotics HST Dual Standard modems very effectively
with my Sun SparcStation and my Mac II at home.  Very-fast downloads are
quite possible... 9600 bits/second or better is not all that uncommon.

Here's what I've learned during my experimentation:

1) It is _not_ sufficient to depend on the error correction supplied by
   the modems... this correction guards against phone-line errors, but
   doesn't protect you against data loss or corruption between a modem
   and a computer.  It's quite possible for a Mac's serial port to drop
   a byte or two, if you're writing large blocks of data to disk
   (interrupts are turned off, and serial-controller overruns can
   occur).  Program-to-program error detection and correction is a
   _must_!

2) Software protocols which have packet-level, stop-and-wait error
   detection and flow control (e.g.  Kermit and XMODEM) don't ride well
   on top of modem-to-modem error detection and correction schemes which
   are also packet-based (e.g. MNP).  This is probably why you've
   noticed such a severe slowdown when you use KERMIT.  

   When your mainframe sends a packet, the modem will break it up into
   one or more modem-to-modem packets (each with its own error-detection
   checksum or CRC).  The receiving modem must receive each such packet
   completely, and validate it, before it can send the first character
   of the packet to your Mac.  This introduces quite a significant
   delay... for example, if the modems are sending 256-byte packets at
   9600 bits/second, there will be roughly a 1/4-second delay introduced
   at the modem-to-Mac end.  There may be an additional delay introduced
   at the mainframe-to-modem end... the modem may wait to suck up a
   whole packet's worth of characters before sending them to its peer
   across the phone line.

   Once the Mac receives the end-of-packet characters from the modem, it
   must recompute the block checksum and send an "ack" packet to the
   modem.  There may be an additional delay introduced before this "ack"
   is sent to the mainframe by the modem at the other end.

   The net result of this extra packet-building and buffering is that
   the sending KERMIT program sits around waiting for quite some time
   before it receives the "ack" and can send another packet.

3) The best way to get around this problem is to use a protocol which
   supports ack-less streaming, or a sliding-window ack protocol.  The
   one I've used most heavily is the ZMODEM protocol, which normally
   operates in streaming mode (no ACKs... just NAKs which say "Back up
   to byte NNN and resend from there").

   ZMODEM can also operate in a sliding-window mode... it has a
   four-packet window which can be set for a total window size of up to
   2k bytes (in the version I use).  I find this mode to be _extremely_
   useful when sending from a Sun to my Mac.  It adds a very useful
   degree of protocol-based flow control... the Sun won't overrun the
   modem's internal buffers if the line gets noisy and the modems must
   retransmit some data.  When the connection is clean (no noise), the
   window is large enough that the Sun can keep the pipeline full of
   data... it gets its ACK back for packet 1 before it has finished
   sending packet 4, for example... and the receiving end _never_ sees a
   pause in the transmission.  It isn't necessary to use XON/XOFF
   software flow control, or RTS/CTS hardware flow control.

   Quite a few Mac telecom programs now support ZMODEM... ZTerm 0.85,
   MicroPhone II, and White Knight come to mind.  I don't know which
   programs (if any) support sliding-windows KERMIT downloading.  On the
   Sun, the "rzsz" package provides the necessary host software.

4) Another way to get around protocol-layering problems is to use a
   modem which supports protocol spoofing... e.g. the Telebit T-1000 or
   TrailBlazer Plus.  These modems are very popular for "spoofing"
   high-speed uucp connections, and they can also spoof XMODEM and
   KERMIT.  They aren't in common use on bulletin-board systems or for
   general-purpose dialin, however.

5) The Macintosh has only one set of "handshake" lines. The commonest
   hookup is to hook the "handshake out" to the modem's DTR pin [so that
   the modem hangs up if you quit from your telecom program] and connect
   "handshake in" to the carrier-detect pin.

   If you want to use hardware flow control, connect "handshake out" to
   the modem's "request to send" input, connect "handshake in" to the
   modem's "clear to send" output, and configure your modem to use
   RTS/CTS handshaking.  Not all modems support this sort of
   handshaking, as it's not an official part of the original RS-232
   specification.  If you use this sort of hookup, make sure that you
   remember to hang up your modem when you log out... the modem will
   _not_ automatically hang up and drop carrier if you quit your telecom
   program or shut down the Mac.




-- 
Dave Platt                                             VOICE: (415) 493-8805
  UUCP: ...!{ames,apple,uunet}!coherent!dplatt   DOMAIN: dplatt@coherent.com
  INTERNET:       coherent!dplatt@ames.arpa,  ...@uunet.uu.net 
  USNAIL: Coherent Thought Inc.  3350 West Bayshore #205  Palo Alto CA 94303

 8-Jun-90  0:51:03-GMT,425;000000000000
Date: Thu, 7 Jun 1990 17:51:02 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: Marek Lugowski <marek@iuvax.cs.indiana.edu>
Subject: Re: HST modems, fellow user's comments 
In-Reply-To: Your message of Thu, 7 Jun 90 18:46:11 -0500 
Message-ID: <CMM.0.88.644806262.psz@sumex-aim.stanford.edu>

Thanks for your note.  I'll check out ZTerm, and (if/when more responses
arrive) summarize to the net.  --Pete Szolovits
 8-Jun-90  0:53:30-GMT,1998;000000000011
Return-Path: <mah@uhccux.uhcc.hawaii.edu>
Received: from uhccux.uhcc.Hawaii.Edu by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA18156; Thu, 7 Jun 90 17:53:29 PDT
Received: by uhccux.uhcc.Hawaii.Edu (5.61/Ultrix3.1)
	id AA06407; Thu, 7 Jun 90 14:51:10 -1000
Date: Thu, 7 Jun 90 14:51:10 -1000
>From: Michael Hoffhines  <mah@uhccux.uhcc.hawaii.edu>
To: Peter Szolovits <psz@sumex-aim.stanford.edu>
Subject: Efficient use of Courier HST modems 
Message-Id: <CMM.0.88.644806269.mah@>

Peter in a recent digets you write -
 
>I have been frustrated by my inability to squeeze optimal performance out of
>our US Robotics Courier HST modems with any of the communication packages I
>have tried (or heard of) and wondered if anyone has found a reasonable
>solution to the identical problem.
I do not believe that your Courier is at fault. Although I have no experience
with the Courier HST, I have experienced similar levels of frustration with
Kermit using a number of different implementations and decided - as you 
apparently have - that the protocol is not all that efficient.

Recently, I have been using White Knight 11.x (aka Red Ryder) and the zmodem
protocol to nearly double my file transfer efficiencies. With kermit, I 
would routinely operate at around 50% of theoretical max and with zmodem that
goes close to 98%. In the sumex archives, there is a unix zmodem implementation
that I was able to compile for our vax with little difficulty. It was worth
all the effort.

Hope this helps and good luck.

- Mike

>-----------------------------------------------------------------------------<
> Michael Hoffhines                    | INTERNET: mah@uhccux.uhcc.Hawaii.Edu <
> ICS Department                       | BITNET: man@uhccux.BITNET            <
> University of Hawaii                 |
>-----------------------------------------------------------------------------<
> "Hey, hey, hey. Don't be mean." B. Bonzai
>-----------------------------------------------------------------------------<

 8-Jun-90  0:55:55-GMT,474;000000000000
Date: Thu, 7 Jun 1990 17:55:55 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: dplatt@coherent.com (Dave Platt)
Subject: Re: High speed modem use with Mac telecom packages 
In-Reply-To: Your message of Thu, 7 Jun 90 16:42:14 PDT 
Message-ID: <CMM.0.88.644806555.psz@sumex-aim.stanford.edu>

Thanks for your thoughtful and helpful comments.  I'll take a look at the
zmodem option, and summarize your response to the net when I get more replies.
--Peter Szolovits
 8-Jun-90  0:57:56-GMT,465;000000000000
Date: Thu, 7 Jun 1990 17:57:55 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: Michael Hoffhines <mah@uhccux.uhcc.hawaii.edu>
Subject: Re: Efficient use of Courier HST modems 
In-Reply-To: Your message of Thu, 7 Jun 90 14:51:10 -1000 
Message-ID: <CMM.0.88.644806675.psz@sumex-aim.stanford.edu>

Thanks for your note.  I'll try out the zmodem approach, and include your
response in a summary posting.  So far, everyone likes zmodem.  
--Peter Szolovits
 8-Jun-90 12:38:58-GMT,3112;000000000011
Return-Path: <CEBESS%KOESS.gm@hac2arpa.hac.com>
Received: from hac2arpa.hac.com by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA29093; Fri, 8 Jun 90 05:38:56 PDT
Received: by hac2arpa.hac.com (5.61++/SMI-DDN)
	id AA18280; Fri, 8 Jun 90 05:37:23 -0700
Date: Fri, 8 Jun 90 05:37:23 -0700
Message-Id: <9006081237.AA18280@hac2arpa.hac.com>
Received: by DnaMail (v1.1); Fri Jun  8 05:21:33 1990 PDT
>From: CEBESS%KOESS.gm@hac2arpa.hac.com       (CEBESS)
To: ::ARPA::psz@sumex-aim.stanford.edu
Subject: slow file transfers

I will attempt to address at least one of your problems. Yes, the protocal has
quite a bit to do with file transfer rate. I use White Knight (version 10 of
Red Rider). It supports Zmodem. I have been able to achieve 97% efficiency over
9600 baud file transfers (about 930 char/sec). Sliding windows kermit should
give you similar results. I know that sliding windows Kermit is available for
both the Mac (latest version of Mac Kermit), VMS and DOS machines. You will
normally see the receive or transmit (depending on what you are doing) stay on
continually and the other light flash every few seconds (depending on packet
size).

You have a missconception about a couple of issues about the 19.2 capability of
the modem. It achives 19200 by doing data compression during the transfer. This
is VERY data dependant. You will probably get no compression from a .SIT file.
You should see some level of compression on a .HQX file. The other item is that
if it were 100% efficient you would see about 1920 chars per second because
there is usually 10 bits of data for every byte (stop bit and possible parity).
The only way to get that speed is just a straight ascii dump.

One of the larges hurdles I find is the work and tuning of the system at the
other end. The Mac should be capable of handling the speed (unless you are
doing Multi-finder jobs of size). The system on the other end needs its type
ahead buffers etc set up to handle the job. Also you mentioned the CISCO box
going to Ethernet. This has added another level of packetizing and protocal
that slow down the process. The CISCO box is very dependant on network traffic
because it has no prioritizing method for small packets (your ACKs from
Kermit). You are also right about the fact that a dirty phone line will cause
these modems to go slightly spastic (without loosing any data of course).

I hope this helps you. I would suggest going to Zmodem if you can find it.
Programs that support it are Zterm (available on the net or other services),
WK and I think Microphone II. The smaller packets can take their time coming
back because the transfer does not wait for them. It is available for most
machines now I believe.

Charles E. Bess                 Internet: CEBESS%KOESS.gm@HAC2ARPA.hac.com      
Electronic Data Systems         Dial-8  : 8-360-5646                            
Suite 100C                      AT&T    : (317) 240-5646                        
2601 Fortune Circle East,       FAX     : (317) 240-5622
Indianapolis, IN 46241-5513     CPS     : 72437,3132
                                America OnLine : CEBess




 8-Jun-90 15:00:53-GMT,954;000000000011
Return-Path: <burton@cs.sfu.ca>
Received: from relay.CDNnet.CA by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA00629; Fri, 8 Jun 90 08:00:47 PDT
>From: burton@cs.sfu.ca
Received: by relay.CDNnet.CA (4.1/1.14)
	id AA24490; Fri, 8 Jun 90 07:57:05 PDT
Date:  8 Jun 90  7:31 -0700
To: psz@sumex-aim.stanford.edu
Cc: burton@cs.sfu.ca
Message-Id: <9006081431.AA13785@cs.sfu.ca>
Subject: Re: Efficient use of Courier HST modems

Pete,

...

As to efficient use of courier HST modems, you will get much better
performance with ZTerm.  I get around 1800 char/sec. under good
conditions using a 14.4 kb Courier HST.  StuffIt files, of course, are
slower since less data compression is possible.

I expect you have 20 replies telling you how to get ZTerm and the Unix
side software (sz and rz).  If not, or if you want more information, let
me know.

Warren Burton
burton@cs.sfu.ca

 8-Jun-90 18:01:09-GMT,3059;000000000011
Received: from gargoyle.uchicago.edu by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA05267; Fri, 8 Jun 90 11:00:44 PDT
Received: by gargoyle.uchicago.edu from tartarus.uchicago.edu (5.59/1.14)
	id AA13561; Fri, 8 Jun 90 12:58:23 199
Return-Path: <owens@cs.uchicago.edu>
Received:  by tartarus.uchicago.edu (4.0/UofC4.0x)
	id AA02518; Fri, 8 Jun 90 12:58:22 CDT
Date: Fri, 8 Jun 90 12:58:22 CDT
>From: Christopher Owens <owens@gargoyle.uchicago.edu>
Message-Id: <9006081758.AA02518@tartarus.uchicago.edu>
To: Peter Szolovits <psz@sumex-aim.stanford.edu>
Subject: Courier HST


I, too use an HST and a Macintosh, and have thought about many of the
issues you are raising in your post.  

As for raw speed, you are right that you want hardware flow control.
White Knight 11.0 (the successor to Red Ryder) claims to support
rts/cts flow control.  Of course, that means you will need a different
cable from mac to modem, because the @#$%^ mac serial ports have only
2 handshaking lines -- the interface can be (software) configured to
use them either as DTR/CD (I think?) or as RTS/CTS, but the mapping of
macintosh pins to modem connector pins will depend on which you are
using.  Although I generally like WK11.0 (gets ctrl-space right, for
example) I haven't tried out hardware flow control yet, because I have
not got around to acquiring the proper cable.  I think MicroPhone
supports it as well, but I've never played with it.

But there is another problem, which is that these modems have two
(relevant) modes -- line speed floats independent of modem-to-computer
link speed, or line speed follows modem-to-computer link speed. The
unit at the Unix end is probably configured the latter way, which is
appropriate for normal dialins from dumb terminals/modems.  The
problem with configuring it the former way is that when someone dials
in at some arbitrary baud rate (not using compresssion) and the modem
recognizes this, you want the computer-to-modem link to also be set to
this same baud rate.  The problem with configuring it the latter way
is that as long as the modem at the Unix end is communicating with the
computer at 9600 baud, you won't get much advantage out of the 19.2 KB
modem-to-modem link.  Talk to the datacomm folks at your site about
this issue -- see if you can find a solution to this, and please let
me know!

You still want an error correction communication protocol, because
even though ARQ guarantees the integrity of the modem-to-modem link,
you want to guarantee the complete end-to-end link.  Every protocol
requires some handshaking between packets; the only question is how
much.  I've been using Zmodem, which does suffer from the turnaround
delay you describe.  Somebody ought to write a protocol that uses
multiple buffers at each end and acknowledges packets asyncronously.

Please post any results you get.


Christopher Owens
Department of Computer Science                 1100 East 58th Street
The University of Chicago                      Chicago, IL 60637
owens@gargoyle.uchicago.edu                    (312) 702-2505

 8-Jun-90 18:15:08-GMT,2783;000000000000
Date: Fri, 8 Jun 1990 11:15:08 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: CEBESS%KOESS.gm@hac2arpa.hac.com (CEBESS)
Subject: Re: slow file transfers 
In-Reply-To: Your message of Fri, 8 Jun 90 05:37:23 -0700 
Message-ID: <CMM.0.88.644868908.psz@sumex-aim.stanford.edu>

Thanks for your suggestions.  I have also been directed by a couple of other
people to using zmodem rather than Kermit protocols, and have just started to
use ZTerm instead of my old Kermit.  Indeed, I just tried downloading a long
file and I got 93% line use (according to ZTerm), which is quite good given the
extra indirection of the CISCO box.  I did not realize that there was a new
Kermit with sliding windows for the Mac.

I know that .sit files will not compress, but I would expect typical text files
to compress by nearly 50%, assuming that the compression algorithm is something
like the Lempel-Ziv that is used in Stuffit and friends.  The major problem
I've had even to try this out is the difficulty of hardware handshaking from
the Mac.  As I mentioned in my net message, I don't like XON/XOFF handshakes
because they get in the way of the "real" data I need to send.  Perhaps if comm
programs just turned it on during file up/download, that would be acceptable,
because it would leave me my beloved control keys for Emacs.  Dynamically
setting this stuff is hard, though, because of the complexity of intermediate
nodes.  In my setup, for example, I think (but am not sure) that I would have
to tell the CISCO box to turn on hardware flow control between it and the modem
to be able to use the modem's data compression, and also to turn off its escape
character if I want to be able to send unquoted binary (so ^^ doesn't get
trapped).  In addition, the Mac's serial port seems to have a generic
"handshake in/out" pair of lines in place of the more normal RS-232 port's
multiplicity of signaling lines.  I think the standard Mac-to-modem cable
connects these to DTR and CD or RI (I'm not sure), not to RTS/CTS, which the
HST modems expect to use for hardware flow control.  So just to do the
experiment, I'd have to make a custom cable and modify some Mac comm program,
and then I'd lose the ability to drop DTR to get the modem to hang up, or maybe
even the ability to auto-answer (if it's RI that now uses that other signal
line).

In any case, getting to almost 9600 baud is still a lot better than what I was
getting before.  I still miss flow control because ZTerm (at least) can't quite
really keep up with 9600 (on a IIx), and eventually drops characters.
Do you know if there's any significant difference in the speed of the
various comm programs you mentioned?

Again, thanks for the info.  --Peter Szolovits

P.S.  I'll include your comments in my summary to the net.

 8-Jun-90 18:21:37-GMT,1310;000000000000
Date: Fri, 8 Jun 1990 11:21:37 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: burton@cs.sfu.ca
Subject: Re: Efficient use of Courier HST modems 
In-Reply-To: Your message of 8 Jun 90 7:31 -0700 
Message-ID: <CMM.0.88.644869297.psz@sumex-aim.stanford.edu>

...

Thanks for the info, which you're right has also been suggested by others (4,
not 20).  ZTerm does in fact give me much better transfer rates (about 93% at
9600), but I have not yet been able to figure out how to make it work with the
hardware handshake of the HST modems.  I know (I think) how to set up the HST's
for 19.2K communication with the Mac and 9600 over the phone, and this should
(with the right other settings) get it to use data compression.  However,
neither Zterm nor anything else I know can tickle the RTS/CTS lines the HST is
interested in.  So, it all works only with XON/XOFF, because without flow
control the modem eventually overruns the Mac (it's a IIx, but can't seem to
keep up).  I dislike XON/XOFF for the reasons I mentioned in my net post.  Have
you gotten around this problem?  If so, how?

Thanks for the info.  --Pete

 8-Jun-90 18:35:47-GMT,1108;000000000000
Date: Fri, 8 Jun 1990 11:35:46 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: Christopher Owens <owens@gargoyle.uchicago.edu>
Subject: Re: Courier HST 
In-Reply-To: Your message of Fri, 8 Jun 90 12:58:22 CDT 
Message-ID: <CMM.0.88.644870146.psz@sumex-aim.stanford.edu>

Thank you for your informative note.  You rightly point out the possible
problem for the Unix end; the comm managers at my lab at MIT made the "right"
choice for the HST's; I don't know what the story is at Stanford, but will
check.  The simplest cable fix is probably not a completely new cable but a
short male-female pass-through that just swithest the appropriate sets of
lines.  (Sort of like the null modems that switch 2-3, but without gender
reversal).  This must be easier to make up than a new din-8 to D-25 cable.

At an earlier suggestion, I tried Zterm/sz(Zmodem), and did in fact get about
93% at 9600 baud, because zmodem only sends NAKs and just blasts through as
long as there are no errors.  Now I'm surprised that it's slow for you!

Thanks very much, and I'll post your note to the net.  --Pete Szolovits
 8-Jun-90 18:41:30-GMT,1286;000000000011
Received: from gargoyle.uchicago.edu by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA06511; Fri, 8 Jun 90 11:41:22 PDT
Received: by gargoyle.uchicago.edu from tartarus.uchicago.edu (5.59/1.14)
	id AA14757; Fri, 8 Jun 90 13:39:05 199
Return-Path: <owens@cs.uchicago.edu>
Received:  by tartarus.uchicago.edu (4.0/UofC4.0x)
	id AA03307; Fri, 8 Jun 90 13:39:03 CDT
Date: Fri, 8 Jun 90 13:39:03 CDT
>From: Christopher Owens <owens@gargoyle.uchicago.edu>
Message-Id: <9006081839.AA03307@tartarus.uchicago.edu>
To: psz@sumex-aim.stanford.edu
In-Reply-To: Peter Szolovits's message of Fri, 8 Jun 1990 11:35:46 PDT <CMM.0.88.644870146.psz@sumex-aim.stanford.edu>
Subject: Courier HST 

   Date: Fri, 8 Jun 1990 11:35:46 PDT
   From: Peter Szolovits <psz@sumex-aim.stanford.edu>

   The simplest cable fix is probably not a completely new cable but a
   short male-female pass-through that just swithest the appropriate sets of
   lines. 

I'm not sure that will do it because I think the "standard"
mac-to-modem cable doesn't pass one of the relevant pins at all, and
it has some other pins jumpered together so that the modem always
seees true on one of the relevant lines. (sorry to be so foggy)

Have fun and remember, datacom will only chew up as much of your
research time as you let it.....

 8-Jun-90 23:46:52-GMT,1536;000000000001
Return-Path: <burton@cs.sfu.ca>
Received: from relay.CDNnet.CA by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA16712; Fri, 8 Jun 90 16:46:14 PDT
>From: burton@cs.sfu.ca
Received: by relay.CDNnet.CA (4.1/1.14)
	id AA03878; Fri, 8 Jun 90 16:43:23 PDT
Date:  8 Jun 90 16:16 -0700
To: psz@sumex-aim.stanford.edu
Message-Id: <9006082316.AA19592@cs.sfu.ca>
Subject: Re: Efficient use of Courier HST modems

>	However, neither Zterm nor anything else I know can tickle the
>	RTS/CTS lines the HST is interested in.  So, it all works only
>	with XON/XOFF, because without flow control the modem eventually
>	overruns the Mac (it's a IIx, but can't seem to keep up).  I
>	dislike XON/XOFF for the reasons I mentioned in my net post.
>	Have you gotten around this problem?  If so, how?

I haven't gotten RTS/CTS to work either.  I think I am now running with
no flow control (I would have to check at home to be sure--I have tried
various combinations.)  On the Unix side I feed into a SPARCstation with
a 19.2 k line, and set my Mac to 19.2.  Usually, I can send large files
(sometimes several megabytes) with only a few resends, so I haven't
worried about it further.  If you can set the line at both ends to
operate at the same speed, thing should work okay in practice, even
thought this doesn't seem like a very nice solution.  Please let me know
if you come up with something better and I will try it.

...

Warren

 9-Jun-90 18:26:36-GMT,1440;000000000001
Return-Path: <ELIOT@cs.umass.edu>
Received: from dime.cs.umass.edu by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA00261; Sat, 9 Jun 90 11:26:27 PDT
Received: from vax3.cs.umass.edu by dime.cs.umass.edu (5.61/Ultrix2.0-B)
	id AA08934; Sat, 9 Jun 90 10:38:28 -0400
Message-Id: <9006091438.AA08934@dime.cs.umass.edu>
Date: Sat, 9 Jun 90 10:37 EST
>From: ELIOT@cs.umass.edu
Subject: Efficient use of Courier HST modems
To: psz@sumex-aim.stanford.edu
X-Vms-To: in%"psz@sumex-aim.stanford.edu",ELIOT

    Hello Peter;
    
    With MacTerminal you can do a straight "Send File" or "Recieve File"
    with no error detection or correction protocol.  If the modem
    does a good enough job of error correction then this might work.
    
    With MacTerminal you can do uploads very easily.  Just turn on the
    "Save Lines Off Top" option and tell your Unix box to print the
    file. Quit and use any Mac Text editor to open a *copy* of your
    saved Macterminal file.  Extract the part you want.  (Most text
    editors will correct a saved MacTerminal file.)
    [The word "upload" in the beginning of that Paragraph should have
    been "download"]
    
    With an upload on a Vax I just tell if to copy into a file from
    the terminal, and then use the "Send File.." file command.  However,
    both of these procedures are subject to line noise, since my modem
    has no hardware error handling.
    
    Good Luck
    
    Chris Eliot

10-Jun-90 17:12:07-GMT,2360;000000000011
Return-Path: <@zermatt.lcs.mit.edu,@HARVEY.lcs.mit.edu:tdwu@ZERMATT.lcs.mit.edu>
Received: from lcs.mit.edu (MINTAKA.LCS.MIT.EDU) by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA00992; Sun, 10 Jun 90 10:12:03 PDT
Received: from ZERMATT.LCS.MIT.EDU by mintaka.lcs.mit.edu id aa01775;
          10 Jun 90 13:04 EDT
Received: from LCS.MIT.EDU by ZERMATT.LCS.MIT.EDU via INTERNET with SMTP id 36847; 10 Jun 90 13:04:41 EDT
Received: from HARVEY.LCS.MIT.EDU by mintaka.lcs.mit.edu id aa01682;
          10 Jun 90 13:02 EDT
Date: Sun, 10 Jun 90 13:02 EDT
>From: Thomas Wu <tdwu@zermatt.lcs.mit.edu>
Subject: Courier HST file transfer
To: psz@zermatt.lcs.mit.edu
Cc: tdwu@zermatt.lcs.mit.edu
Message-Id: <19900610170207.1.TDWU@HARVEY.LCS.MIT.EDU>


Peter,

I saw your post on info-mac.  I also have one of the Courier HST 9600
modems from our group, and I get near 100% transmission rate (960
characters per second) when I upload and download to HX.

The trick is to use the Zmodem protocol (the sz and rz commands on
UNIX), which doesn't acknowledge any packets unless there's an error.
The Kermit, Xmodem, and normal Ymodem schemes both acknowledge every
packet, which kills the transmission rate because of the asymmetric
9600/300 baud scheme.  (But Ymodem-G protocol, I think, is also like
Zmodem in that it doesn't acknowledge each packet.)

The software I use is White Knight 11.  Red Ryder 9.4 apparently had
problems with keeping up with 9600 baud screen updating; maybe it also
couldn't keep up with 9600 baud file transfers.  There is a shareware
program, Zterm, on info-mac that also supports Zmodem.

With regard to hardware handshaking, White Knight 11, Smartcom, and
Microphone are supposed to support it, but you need the right cable.
Most cables don't connect the CTS/DTS lines required.  I have an article
>From MacUser which shows how to roll your own cable.  Some specialized
companies also sell the right cable.  I haven't tried getting the right
cable yet, so I don't know if all this will work.

With regard to Emacs, apparently there is a software hack for White
Knight 11.  You can remap keys, so ^S and ^Q get changed to something
the modem doesn't recognize.  Then, on the UNIX end, you also remap the
keys for ^S and ^Q.  Again, I haven't really tried this yet.

Hope this helps; I initially had problems also when I first got the
modem.

Tom

11-Jun-90 15:26:15-GMT,601;000000000000
Date: Mon, 11 Jun 1990 8:26:14 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: Thomas Wu <tdwu@zermatt.lcs.mit.edu>
Subject: Re: Courier HST file transfer 
In-Reply-To: Your message of Sun, 10 Jun 90 13:02 EDT 
Message-ID: <CMM.0.88.645117974.psz@sumex-aim.stanford.edu>

Thanks, Tom.  Your reply is very much in line with the other suggestions I've
gotten, namely use zmodem.  Also, just like you, people think there are ways of
using the 19.2KB feature and eating their cake too, but they haven't quite
gotten around to it.  I'll include your response in my summary and posting.

--Pete
11-Jun-90 20:05:37-GMT,1432;000000000011
Return-Path: <djb@ctc.contel.com>
Received: from ctc.contel.com (turing.ctc.contel.com) by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA22414; Mon, 11 Jun 90 13:05:32 PDT
Received: from hobbes.ctc.contel.com by ctc.contel.com (4.0/SMI-4.0)
	id AA00907; Mon, 11 Jun 90 16:03:27 EDT
Date: Mon, 11 Jun 90 16:03:27 EDT
>From: djb@ctc.contel.com (Dave Bursik  x4497)
Message-Id: <9006112003.AA00907@ctc.contel.com>
To: psz@sumex-aim.stanford.edu
Subject: Re: Efficient use of Courier HST modems


Peter,
I read with interest your note on the Courier modem, as we
have one of them here that I plan to use for high-speed
dialup links (not sure exactly which model we have, just
know that it's 9600).

I wonder if the link between the terminal concentrator
and the host is what's slowing you down (as well as the
load on the host).

If it's possible (in your environment), see if you can
hook one of these modems to a serial port on the host
(temporarily) to see if that reduces your download times.
As a control on the "experiment", try picking a lightly-
loaded machine or at least a time of day when the load
is typically light.

I'd try it myself here, except I only have one modem
right now, so it won't work very well.  :-}

Dave Bursik, Sr. Member of Technical Staff
Contel Technology Center
15000 Conference Center Dr., P.O. Box 10814
Chantilly, VA  22021-3808
E-mail: djb@ctc.contel.com / Voice: (703) 818-4497 / FAX: (703) 818-5484

11-Jun-90 22:39:06-GMT,1367;000000000011
Return-Path: <ROSKAR@jhuvms.hcf.jhu.edu>
Received: from jhuvms.hcf.jhu.edu by sumex-aim.stanford.edu (4.0/inc-1.0)
	id AA29351; Mon, 11 Jun 90 15:39:03 PDT
Date: Mon, 11 Jun 90 18:37 EDT
>From: Veljko Roskar <ROSKAR@jhuvms.hcf.jhu.edu>
Subject: Hardware flow control with a Mac
To: psz@sumex-aim.stanford.edu
Message-Id: <7F2DAF759EFF204F93@JHUVMS.BITNET>
X-Envelope-To: psz@sumex-aim.stanford.edu
X-Vms-To: IN%"psz@sumex-aim.stanford.edu"

Do you have the properly configured cable for hardware flow control?
The standard Mac modem cable allows flow control in only one direction,
which means you have to adapt it to allow RTS/CTS flow control.  

If you need to know how to do this, reply and I'll dig it up.  The only
catch is that you sacrifice DTR, but you can tell the modem to ignore DTR or
better configure the cable so the modem thinks DTR is always on.  That way you 
can switch between communications programs without having the modem
disconnect you.

ZTerm 0.85 supports hardware flow control and the Zmodem protocol works 
really nice with a Unix machine.

Hope this helps.

Best regards,

Veljko Roskar				  BITNET:  roskar@jhuvms.bitnet
Department of Chemical Engineering	INTERNET:  roskar@jhuvms.hcf.jhu.edu  
   The Johns Hopkins University		    UUCP:  !mimsy!aplcen!jhunix!roskar
     Baltimore, Maryland 21218		    tel.:  301-338-7054
	      U.S.A

12-Jun-90  0:06:42-GMT,908;000000000000
Date: Mon, 11 Jun 1990 17:06:42 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: djb@ctc.contel.com (Dave Bursik x4497)
Subject: Re: Efficient use of Courier HST modems 
In-Reply-To: Your message of Mon, 11 Jun 90 16:03:27 EDT 
Message-ID: <CMM.0.88.645149202.psz@sumex-aim.stanford.edu>

Thanks for your suggestion.  I tried that experiment, and have also noted
fluctuations in efficiency based on both network and host load, but none of
these makes enough of a difference.  I think the fundamental problem is the
acknowledgement protocol used by Kermit and many of the others.  Numerous
people who responded suggested zmodem, which does in fact seem to do much
better, at least getting very close to the 9600-baud capabilities of the HST.
It's much harder to achieve 19.2kb, for reasons I hope to summarize to the net.
Even the improvement I've seen to 9600 is great, though.  --Peter Szolovits

12-Jun-90  0:15:55-GMT,949;000000000000
Date: Mon, 11 Jun 1990 17:15:54 PDT
>From: Peter Szolovits <psz@sumex-aim.stanford.edu> 
To: Veljko Roskar <ROSKAR@jhuvms.hcf.jhu.edu>
Subject: Re: Hardware flow control with a Mac 
In-Reply-To: Your message of Mon, 11 Jun 90 18:37 EDT 
Message-ID: <CMM.0.88.645149754.psz@sumex-aim.stanford.edu>

Thank you for your comments.  I have received the same suggestion from a few
others, and indeed zmodem/zterm work fine, at least up to 9600 baud.  I
haven't been able to try at higher speeds because I don't have the right cable,
and because our terminal concentrator is not set up for higher baud rates.  If
it's not too much trouble, the pin assignments for the cable that works would
be helpful.  As I see it, it looks like Zterm actually manipulates what it
thinks is DTR, so presumably the cable needs to map whatever line that is
normally in the DIN-8 output on the back of the Mac to the right pin for CTS or
RTS at the Modem.

--Peter Szolovits